Skip to content

fetch: make readAllBytes sync#4349

Open
Uzlopak wants to merge 15 commits intomainfrom
readAllBytes-sync-v2
Open

fetch: make readAllBytes sync#4349
Uzlopak wants to merge 15 commits intomainfrom
readAllBytes-sync-v2

Conversation

@Uzlopak
Copy link
Copy Markdown
Contributor

@Uzlopak Uzlopak commented Jul 18, 2025

Because we cant access the ReadableStreamDefaultReader logic inside node core to provide our own reader with the specified logic as mentioned in the spec. Before readAllBytes is async, which is confusing because we provide two callbacks to it. Now it reads more natural for me.

Status

Copy link
Copy Markdown
Member

@KhafraDev KhafraDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no reason to change it

Copy link
Copy Markdown
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allocates way less promises and I expect to improve performance.

I don't really understand how this work tho. What if part of the body is not available?

@Uzlopak
Copy link
Copy Markdown
Contributor Author

Uzlopak commented Jul 19, 2025

If body is null or invalid then it would break already in an earlier stage in the function which calls readAllBytes, because we call getReader() on the body which can throw an error. We expect in readAllFiles the generated Reader.

The only place we can error is when the reader is locked, so that read will throw. Other than that i dont see any code path possible to result in an error.

@Uzlopak
Copy link
Copy Markdown
Contributor Author

Uzlopak commented Jul 19, 2025

LOL, I actually found a bug in the sri code. I was wondering why setting the rejection handler in readLoop is not working.

@mcollina
Copy link
Copy Markdown
Member

What about if a large file is transmitted? Is fetch already buffering all of that in memory?

@Uzlopak
Copy link
Copy Markdown
Contributor Author

Uzlopak commented Jul 19, 2025

Only if you use one of those methods from the bodyMixing, like .text()``,.json() and so on. If you know that you expect a huge file you should stream it anyway.

For streams you have that logic directly in mainFetch called incrementalReadBody or so. There we use alot of magic like async generators.

@gurgunday
Copy link
Copy Markdown
Member

Can you bench it somehow? I think it's a pretty good change if there's a significant improvement

@Uzlopak
Copy link
Copy Markdown
Contributor Author

Uzlopak commented Aug 15, 2025

@gurgunday

What I need is a way to count the generated Promises.

@gurgunday
Copy link
Copy Markdown
Member

I use https://github.com/bengl/count-promises for that

Copy link
Copy Markdown
Member

@gurgunday gurgunday left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Copy Markdown
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Copy Markdown
Member

@Uzlopak this now conflicts. can you use #4349 (comment) to count the promises?

@KhafraDev this seems to allocate way less promises than async/await combo.

@Uzlopak Uzlopak force-pushed the readAllBytes-sync-v2 branch from e7e6644 to 36cf917 Compare August 22, 2025 20:30
@Uzlopak
Copy link
Copy Markdown
Contributor Author

Uzlopak commented Aug 22, 2025

My change reduces according to the "test" that the amount of promises is reduced by 1 per fetch call

@KhafraDev
Copy link
Copy Markdown
Member

1 less promise?

@Uzlopak
Copy link
Copy Markdown
Contributor Author

Uzlopak commented Aug 23, 2025

yes, 1 less promise. We create about 50 promises for a fetch call.

See the other PRs regarding removing async/promises. Maybe we can remove overall about 10 promises.

Comment thread test/fetch/count-promises.js Outdated
Copy link
Copy Markdown
Member

@gurgunday gurgunday left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just my 50c but I think this and other PRs that remove unnecessary promises are ultimately beneficial and they add up

Copy link
Copy Markdown
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Copy Markdown
Member

@KhafraDev can you take another look? I don't see how this is problematic

@trivikr
Copy link
Copy Markdown
Member

trivikr commented May 4, 2026

This needs a rebase, so that latest tests can run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants